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ABSTRACT 

This thesis illustrates the use of simulation techniques to evaluate the satisfaction 
of suitability requirements for a mobile platform carrying payload (for example, an 
Unmanned Aerial Vehicle with sensors) on a military mission (surveillance or 
reconnaissance). The Institute for Defense Analyses, in support of Director, Operational 
Test & Evaluation (DOT&E), recently developed a simulation to assist in the analysis of 
the PREDATOR Unmanned Aerial Vehicle. That simulation has been extended to make 
it more applicable to a variety of platforms, and the extended simulation has been 
incorporated into the Military Aircraft Sustainability Simulation (MASS). The primary 
output from the simulation is an estimate of Effective Time On Station (ETOS), as that 
depends on platform subsystem reliability and the maintenance resources allocated. 
ETOS is the long-run percentage of time that the region under surveillance is being 
covered by at least one operating platform. An analytical model for a single platform 
also has been developed to augment and assist in verifying the MASS. This thesis shows 
that MASS can be an invaluable tool for evaluating a platform’s suitability for a mission. 
The simulation can assist during the acquisition process, when the government must 
decide whether to buy a platform, and the simulation can assist in determining the most 


effective way to deploy such platforms once they are in use. 





THESIS DISCLAIMER 


The reader is cautioned that computer programs developed in this research may not have 
been exercised for all cases of interest. While every effort has been made, within the 
time available, to ensure that the programs are free of computational and logic errors, 
they cannot be considered validated. Any application of these programs without 


additional verification is at the risk of the user. 
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EXECUTIVE SUMMARY 


The suitability of a platform to carry out a mission is a very important factor in 
determining the success of that mission. Simulations can be used to explore scenarios for 
testing the suitability of platforms where there is no field test data, and to extrapolate and 
interpret such data when it becomes available. This thesis illustrates the use of 
simulation techniques to evaluate the suitability of a mobile platform carrying payload 
(for example, an Unmanned Aerial Vehicle (UAV) with sensors) for a military 
surveillance or reconnaissance mission. 

Mr. Joseph A. Post and Dr. Catherine W. Warmer of The Institute for Defense 
Analyses, in support of Director, Operational Test & Evaluation (DOT&E), recently 
developed a simulation to assist in the analysis of the PREDATOR Unmanned Aerial 
Vehicle. That simulation has been extended and generalized to make it applicable to a 
variety of platforms, and the extended simulation has been incorporated into the Military 
Aircraft Sustainability Simulation (MASS) program. MASS is modified and explored 
extensively in this thesis. 

The primary output from the simulation is Effective Time On Station (ETOS). 
ETOS is the long-run percentage of time that the region under surveillance is being 
covered by at least one operating platform. Other measures of effectiveness can be 
computed as well. 

An analytical model for a single platform has been developed to augment and 
assist in verifying the MASS. The results from the analytical model confirm that, under 
certain specific conditions, MASS accurately estimates ETOS for a single platform. This 


provides some assurance that the simulation program is valid. 
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Many analyses are done using the simulation. The first analysis using MASS 
confirmed one of the findings from the Predator analysis; it is generally important to base 
the platform as close to the surveillance region as possible. Such basing minimizes the 
likelihood of a failure occurring during ingress, and minimizes the time for replacement 
platform to arrive on-station when needed. 

One of the more interesting findings was the simulation’s sensitivity to time-to- 
failure distributions. The common assumption made in many system reliability models 1s 
that the times between failures are exponential. Based on analysis in this thesis, such an 
assumption can be overly optimistic. This finding suggests the need for actual field tests 
to better understand the operational suitability of the system. 

The human factor in the operation being simulated is discussed at length. There 
are assumptions made in the simulation that affect the results, but that will not necessarily 
hold true in actual operations. Maintenance of the platforms is a large factor in these 
assumptions. Repair time calculations in serial are pessimistic when one considers that 
more than one repair can be completed at a time. The squadron will also have decisions 
to make on the level of repair with iia platform is allowed to launch. In some cases 
it is helpful to launch platforms at less than 100% capability. 

Deployment length is also a factor in the suitability analysis. Long deployments 
are used for analysis to ensure the steady-state ETOS is attained. However, shorter 
deployments may be more realistic, even though it is demonstrated that the variability of 
the results is greatly increased for those short deployments. Such variability should be 


accounted for when planning deployments. 
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MASS can be an invaluable tool in the evaluation of a platform’s suitability for a 
mission. The simulation can assist first during the acquisition process, when the 
government must decide whether to buy a platform, and the simulation can later assist in 


determining the most effective way to deploy such platforms once they are in use. 
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I. INTRODUCTION 
A. GENERAL 

One of the most important questions facing a decision-maker during mission 
planning is how to maximize the probability of a successful mission. Given that a variety 
of system designs (e.g. platform and sensor payloads) are possible choices for a mission, 
the ability to evaluate each system’s operational suitability and effectiveness for a 
particular mission will largely decide which platform is chosen for a mission, and how 
that platform is configured and used. Computer simulation is often the most cost- 
effective way to estimate a platform’s suitability. Simulations can be used to explore 
scenarios for which there is no field test data, and to extrapolate and explore such data 
when available. 

There are two critical times when a platform’s suitability is analyzed. The first 
occurs during the acquisition process, when the government must decide whether or not 
to buy the platform and ultimately adopt it in the field. Minimum performance 
requirements are normally established for a platform based on the anticipated usage, and 
it is Important that these include total mission suitability and effectiveness. It is the job 
of the government (the Test & Evaluation community) to ensure that the platform it is 
purchasing satisfies these requirements. The second critical time is when platforms and 
the support assets are deployed. The unit commander must decide how many platforms 
to deploy, where they should be based, and what type of support they will need to be 
effective. 

This thesis illustrates the use of simulation techniques to evaluate the satisfaction 


of suitability requirements for a mobile platform carrying payload (an Unmanned Aerial 








Vehicle with sensors) on a military mission (surveillance or reconnaissance). With some 
modifications the same simulation can be used for evaluating other military systems. 
B. PROBLEM DESCRIPTION 

There are many factors to consider when analyzing the suitability of a platform. 
The person evaluating such an asset must be able to look at these factors and determine 
whether the platform can satisfactorily complete its mission, and also make future 
contributions. — 

Among the many questions that arise are the following: 

How many platforms are needed for the particular deployment in order to achieve 

adequate area coverage? 

How far away from the target area can the platform be effectively based? 

Can the platform be refueled in-flight, perhaps near the surveillance region? 

How does the platform’s reliability affect the mission? 

What type of maintenance support is needed? 

How should the maintenance concept of operations be structured? 

What type of logistics support is needed? 
The answers to such questions are critical inputs for a decision-maker tasked with 
assessing the suitability of surveillance platforms for particular deployments. Simulation 
methodology is useful for providing answers. 
C. SCOPE OF THESIS 

The purpose of this thesis is to produce a simulation tool to assist in answering 
such important operational questions by providing information about platform suitability 


for various missions; suitability 1s meant in the technical Test & Evaluation sense of 











DOD 5000.2. By changing input parameters, trends can be detected which allow those 
platform characteristics most critical to a platform’s successful mission completion to be 
identified. Identification of critical platform characteristics in this manner also allows 
required operations capabilities to be specified for future platforms to ensure that they are 
capable of successfully performing their intended missions. 

Most of this study uses event-step Monte Carlo simulation. Analytic 
(mathematical) models are used in some cases to assist in validating the simulation 
program, and sometimes to exercise its options quickly. Plausible alternative 
distributions for the times to failure and repair are tested for their effect on results, as are 
alternative maintenance and operational strategies. Changes in the distributions of time 
to failure can importantly affect overall system performance; these sensitivities can be 
efficiently discovered by simulation. They eieseet the importance of actual field tests to 
verify predictions. 

The simulation of this thesis addresses several important operational issues, but 
there is definitely room for further investigations; some are on-going. Further 
considerations are examined, and suggestions made for continuing work on this general 
important problem. 

D. THESIS STRUCTURE 

This thesis has five chapters. The first chapter is an introduction to the problem 
and the content of the thesis. The second chapter covers the background of the problem 
being addressed in order to give the reader a better understanding of the motivation for 
the thesis work. The third chapter focuses on the simulation itself and explains how the 


program works so that it can be used by people with different backgrounds. The fourth 














chapter examines the analytic models used to supplement the simulation. The fifth 
chapter summarizes the results and the conclusions that can be drawn from them. It also 


suggests areas for other applications and for further research. 





Il. BACKGROUND 

The Director, Operational Test & Evaluation (DOT&E) is responsible for 
“reviewing and analyzing the results of OT&E conducted for each major DoD acquisition 
program” (See the web site: http://www.dote.osd.mil/about.html). These results are used 
to make recommendations to the Secretary of Defense and Congress regarding 
acquisition programs and to “confirm operational effectiveness and suitability of the 
defense system in combat use” (http://www.dote.osd.mil/about.html). The Institute for 
Defense Analyses (IDA) provides technical assistance to DOT&E in carrying out its 
mission. 

One of the acquisition programs recently studied by simulation was the 
PREDATOR Unmanned Aerial Vehicle (UAV). As part of the assessment of this 
system, Mr. Joseph A. Post and Dr. Catherine W. Warner (IDA) developed a model 
written in EXTEND™ to assist with the analysis of the Predator system. The model is a 
discrete-event simulation designed to help determine the Predator’s suitability for a 
continuous surveillance mission (Post and Warner, 1997). The measure of effectiveness 
chosen and studied is the long-run percentage of effective time on station (ETOS). The 
ETOS measure is also used throughout this thesis as the main measure of effectiveness 
(MOE), and is defined as the (estimated) expected number of hours with at least one 
operational platform on station divided by the total length of the deployment. Other 
measures are also possible and instructive. 

Post and Warner used their model to perform a sensitivity analysis of various 
operationally important factors, such as the long-run number of UAVs operationally 


available, and the squadron’s ability to complete maintenance. They showed that the 








ETOS is very sensitive to the Predator’s range to the target area. They also showed that 
having to order spare parts instead of keeping them on hand could significantly secreace 
the ETOS. These results made the model so useful and instructive that DOT&E 
expressed interest in using similar models on other projects. 

Dr. Kenneth D. Pendergast of IDA and LT Stoneman have extended the Post- 
Warner model to make it more applicable to a variety of platforms, and have incorporated 
| the extended model into the Military Aircraft Sustainability Simulation (MASS ) 
program (Pendergast, 1998). This thesis discusses the extended MASS model, compares 
MASS results to those from a simple analytical model, and further enhances MASS to 
analyze factors that affect the ability of a flying unit to perform operationally significant 
surveillance tasks. 

Standard Scenario 

The basic scenario for the model is that a squadron of surveillance platforms, here 
airframes with a sensor payload, has been assigned to maintain constant presence in a 
(possibly remote) region. The airframe is susceptible to in-flight failures, which will 
sometimes necessitate a return to base, and the immediate launch of another platform in 
response. In some cases, certain malfunctions can be repaired in-flight. The platform 
can remain airborne for a limited time, although in-flight refueling may be a possible 
option. The question of interest is: does the squadron have the ability to maintain the 
required surveillance level, given the distance to the region to be covered, the number of 
platforms in the squadron, and the available capabilities to provide restoration and repair, 
namely the sustainment assets and concept of operations? The simulation provides a 


logical and economical answer to this question, but depends on plausible and convenient 











assumptions based on anticipated reality. Field operational tests are recommended to 
reassure the system acquisition decision makers that simulation results can be trusted: 
test results may call for model modifications, revealing initial model assumption 
inaccuracies that require correction. These can then be incorporated into the model for 
new runs. The interplay of modeling and field testing reveals options and minimizes 


surprises when the platform system is produced or released for field usage. 











Il. MILITARY AIRCRAFT SUSTAINABILITY SIMULATION (MASS) 
MODEL 
A. GENERAL 
MASS is a Monte Carlo, object-oriented, discrete event simulation written in C++. 
For each repetition of the simulation, a fictitious but representative deployment (many 
flights of the moun over a time period with intervening maintenance periods) is 
created. The current implementation of the simulation can accommodate many 
platforms, but they must all be of the same type (at present). There can be multiple repair 
stations, but they must all perform the same types of maintenance. Each platform is 
permitted to have multiple failures during a flight; these can occur in the different 
subsystems that make up the entire platform system. 
B. MASS MODEL INPUTS 
The inputs for the simulation and a brief description are as follows: 
b-1: Number of Platforms — The number of platforms in the squadron. For this 
study, all platforms in a squadron are assumed to be identical, and the number 
remains constant; platform attrition is not currently modeled, but its effect can be 
included with little difficulty 1f desired. 
b-2: Number of Ground Maintenance Paths — The number of platforms that can 
have ground maintenance performed on them at the same time. This maintenance 
capability can be limited by the number of maintenance personnel or available 
workspace. However, one platform can not occupy more than one path, and no more 
than one platform can be serviced simultaneously by a path. If there are fewer 


maintenance paths than platforms, delays to begin service can occur. 








b-3: Maximum Number of Platforms Flying — This input denotes the number of | 
platforms actually in the air at one time. The present study emphasizes UAVs, that 
are limited by their requirement for ground control. However, if a squadron had 
insufficient pilots or aircrew this input could be used to simulate the effect of that 
shortage also. 

b-4: Ingress/Egress Time (in hours) — The length of flight from home base 
(hangar) to mission location. Distance must be converted to time in the simulation. 
Egress time is still assumed equal to ingress time if the platform has a failure and 
must return home from the refueling orbit. Realistically, these times may differ 
systematically because of wind, and haphazardly/randomly because of route or 
altitude changes. The program can be changed to recognize these details. 

b-5: Scheduled On-Station Time (in hours) — The maximum length of time the 
platform is scheduled to remain on station, provided there are no mission-terminating 
failures. This depends on the platform and aircrew endurance, fuel capacity, and 
environmental conditions. 

b-6: Flight Time (in hours) Before Refuel — The length of platform flying time 
before there is need to refuel. This is the amount of time until the operation 
controller wants the Platform to be at the refueling orbit; it may not necessarily 
depend directly on the Platform’s endurance. 

b-7: Transit Time (in hours) to Refueling Orbit — The length of platform flying 
time from mission orbit to refueling track. The platform is not on-station during this 


time. 
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b-8: Time (in hours) Required to Refuel Platform — The length of time that the 
platform is expected to stay at the refueling orbit. This includes the time required to 
actually find the tanker as well as refuel, and return to éoveraee mode. The platform 
is not on-station during this time. 

b-9: Probability That a Failure is In-Flight Repairable — It is assumed that some 
failures can be repaired by maintenance personnel while the platform is airborne. 
The long-run fraction of failures that can be repaired while the platform is airborne 1S 
used in the simulation as the probability that the failure is in-flight repairable. 
Obviously an unmanned platform does not have this capability although 
automatically switched-in redundant (sub) systems is a design option (the switch 
itself may be failure-prone). The number is set to 0 for UAVs and any other 
platforms that do not have this capability. 

b-10: Mean Time (in hours) Between Mission-Affecting Failures — The expected 
time between failures, the occurrence of which prevent the platform from performing 
its mission (in this model such failures do not cause aircraft/platform loss, e.g. crash). 
Note: A mean may not alone be sufficient to summarize and model random time 
between system failures, nor may the conventional convenient and simple 


exponential distribution. Such is used here for illustration. Sensitivity tests of the 


“- exponential distribution assumption have been performed in this thesis, and they 





show that an important sensitivity may be present. 
b-11: Mean Time (in hours) Between Non-Mission-Affecting Failures — The 
expected time between failures that do not directly affect the Platform operational 


status. These failures will, however, extend the amount of repair time needed on the 





ground and thus eventually reduce ETOS. They may, for example, be failures of 
subsystems backed up by redundant copies. The times between failures are assumed 
to have an exponential distribution. Note: See comment b-10. 

b-12: Mean Time (in hours) for Ground Repair — The expected time for ground 
maintenance personnel to diagnose and repair each reported failure. Note: The 
entire dimuon of repair time, by repair type, is actually required, except when the 
deployment is very long. This thesis uses the exponential distribution for illustration. 
Repair times depend upon which subsystems have failed, and on the decision 
whether to repair them during the present deployment. 

b-13: Mean Time (in hours) for In-Flight Repair — The expected time for 
maintenance personnel (if present) to diagnose and repair a failure while in-flight. 
The repair times are assumed to have an exponential distribution. In some cases, 
diagnosis and repair in-flight will not be possible before the platform is scheduled to 
return to base, causing mission termination. The possibility of mis-diagnosis is not 
considered, but the effect can be simulated and should be the subject of field test. 
Note: See comments b-10, b-12. 

b-14: Flight Time (in hours) Between Scheduled Maintenance Actions — The 
flight time duration between mandatory maintenance periods. Initially, a constant 
decision variable. There is no attempt to relate the occurrence of random failures to 
the scheduled maintenance actions. 

b-15: Scheduled Maintenance Service Time (in hours) — The time for ground 


maintenance personnel to complete a scheduled maintenance action. This value is 

















represented as a constant, since the required maintenance actions are nearly the same 
each time. 

b-16: Deployment Duration (in hours) — The length of the deployment (e.g. part or 
all of an operation or campaign) that is being simulated. Each platform/aircraft will 
typically accomplish several/many duty cycles (out, on-station, back, 
restoration/repair; out, ...) during such a deployment. 

b-17: Logistics Delay Time (in hours) — The time to obtain all parts needed for 
maintenance actions for one Platform from the maintenance department. This time is 
added to the repair time for the failure. The delay times are assumed to have an 
exponential distribution. This input in the present simulation assumes that the parts 
are present at a forward maintenance level. This may not occur. “Cannibalization” 
of existing failed systems can occur: this would reduce the number of platforms 
available; its effect can be studied by simulation but this is not done in this paper. 
b-18: Platform Turn-Around Time (in hours) — The time required to launch a 
platform, measured from the moment it returns from its previous cycle. Even if a 
platform is failure-free there is a minimum required time to refuel and prepare for the 
next launch. The time is assumed to be a constant. 

b-19: Time (in hours) Between Simulation Results Output — The time of 
reporting of model outputs. Often, but not always, this will be the length of the 
deployment just to record the final results, such as the effective time on station. Such 
time will vary randomly across individual simulations. The simulated distribution of 
such outputs, for instance its mean, provide information useful to deployment 


decision makers. 
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b-20: Number of Simulated Deployments (Replications) — This is the number of 
model replications to be run under the given conditions. The number depends on the 
confidence levels, and widths of confidence intervals, required by the analyst for the 
estimates of the measures of effectiveness (MOEs) obtained by simulation: the more 
precision required, the more replications are needed. 

C. MASS MODEL FLOW 


The flow chart in Figure 1 is a general description of the sequence of events that 


each Platform in the model goes through. 


ON ALERT 


MAINT / TURN 





e While the platform is in the air, it may have failures that shorten the time until it 
egresses, but the sequence of steps remain the same 
e A platform can refuel in-flight from any stage if scheduled to do so 


Figure 1. MASS Sequence of Events 














The first step in running the simulation is to input the desired parameter values for 
the appropriate mission and platform. These have been discussed already, but it is 
important to address assignment of a value for a parameter that is not applicable for a 
given platform. For example, some platforms do not have an in-flight refueling 
capability. To keep this from affecting the simulation, a sufficiently large number can be 
input for the Flight Time Before Refuel so that the Platform must land before it tries to go 
to the refueling orbit. This method is used for any times between, or times until, event 
input. There is no default value. However, for the simulation runs in this thesis, 50,000 
hours was the standard input for events that were not desired. For inputs that affect the 
length of an event, such as Transit Time to Refueling Orbit, 0 is used as the input for 
events not used (if a platform is being refueled on-station). 

The simulation begins with an order to iaitich the first platform. The time of the 
platform’s first failures, mission-affecting and non-mission-affecting, and the scheduled 
on-station, off-station, refueling, and land times are calculated. The failure can be a 
mission-affecting or non-mission-affecting failure depending on the user inputs and the 
independent random number draws to iia the times. There is no first failure if its 
nominal time exceeds the length of the flight. 

The platform immediately goes to the Ingress phase of the flight. From this phase on 
the platform is susceptible to failures, or need to refuel, at any time according to 
schedule. If the platform has a mission-affecting failure, it requests that another platform 
be launched, and it returns to base. If the platform has a non-mission-affecting failure, it 
continues to its station. Non-mission-affecting failures do not affect the conduct of the 


platform. They do, however, increase the amount of time needed to repair the platform 
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once it returns to base. Also, some types of platforms allow for in-flight repairs. 
Whenever a failure occurs, the time until the next failure 1s automatically calculated. The 
Platform can refuel during ingress. If the return time (from refueling) is after the start of 
the scheduled on-station time, then the platform assumes On Station status. 

The next phase for the platform is On Station. The platform remains there until or if 
it next experiences a mission-affecting failure, is sent to refuel, or it has reached the 
original return time, whichever comes first. A replacement, if available, will be launched 
SO as to arrive on station as the first platform is leaving to return to base. A replacement, 
if available, is also launched when an On Station platform experiences a mission- 
affecting eailbie A replacement platform does not arrive to provide coverage while the 
first platform is refueling. 

Some platforms are sent to the Refueling phase, in which case they must transit to 
the refueling orbit. Such a platform spends the required amount of time refueling and 
then transits back to be on-station unless the scheduled on-station time has been reached. 
Failures can occur during this time. If the platform is required to return to base during 
the refueling phase because of a failure or flight time endurance limit, the egress time is 
assumed to be the same as if it were returning directly from on-station position. 

When it is time for a platform to return to base, it is said to be in the Egress phase. 
Platforms are allowed to refuel during egress. Failures that occur during this phase may 
only affect the maintenance time on deck, since the failure occurred during Egress rather 
than while on-station. Once again, catastrophic events leading to platform loss are not 


modeled. 
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Unless a platform has returned early because of the occurrence of a mission-affecting 
failure, it enters the Land phase at the scheduled time. If it has any failures, it looks for 
an open maintenance facility. If maintenance is available, the platform enters the 
Maintenance phase. If all maintenance stations or paths are busy then the platform must 
wait. There is currently no prioritization of maintenance; repair is “first-come, first 
served” at present, although “shortest task first” could be more effective if the shortest 
task were identifiable. Once the repairs are completed, or if none are needed, the 
platform enters the Turn phase where it is prepared for its next flight. After it has been 
turned, the platform is ready for Launch and waits until it is needed. 

These basic processes are repeated for all platforms for the length of the deployment. 
D. MASS MODEL ASSUMPTIONS 

Certain assumptions were made for the original version of MASS to simplify the 
model. Some of the key assumptions are: 


e Planned continuous (24 hour) coverage of region (some unrealized because of 
failures) 

No attrition or loss of platforms 

No weather-related effects 

No ground aborts 

All platforms in unit have identical failure rates and repair times (or, 
generally, failure and repair time distributions) 

Failure rates do not change during different mission phases 

Failure process stable (no latent defects to occur, be removed) 

All in-flight repairs are successful (if possible; not for UAVs) 

Platforms may not launch with any uncorrected failures (subject to 
modifications if failure is unrelated to mission) 

Maintenance performed on a First-come, First-served basis 

No functional check flights 

Scheduled maintenance service time is constant 

No crew limitations (air or maintenance) 

No ground system failures 

Platforms are as good as new when launched (e.g. the random time to failure 
is redrawn from a fixed distribution) 





All of the above can be relaxed or changed. Several are strong candidates for change 


to provide more realism or to perform sensitivity tests. In chapter V of this thesis, some 


of these assumptions are modified to analyze their effect on platform suitability. 


E. MASS MODEL OUTPUT 


fe 


Output from the simulation includes the following: 

Effective Time On Station (ETOS) — The mean percentage of time that the region 
under surveillance is being covered by at least one operating platform for a given 
deployment duration. 

Platform availability — The mean percentage of time a platform is mission capable 
(not necessarily over the region). This is presented as one number that is the average 


availability of all platforms during the deployment. This will vary with the 





deployment duration and with initial conditions, unless the deployment is very long. 
This is related to ETOS. 

Sortie generation rate (sorties per day) — The average number of launches during 
each 24-hour period. 

Total sorties during deployment (sorties) - The total number of launches for the 


entire deployment. 


For each of the outputs, a mean, and a confidence interval for the mean, are 


calculated. All confidence intervals in this thesis are calculated based on the assumption 


of approximate normality of the simgle mean. These normal confidence intervals were 


checked against the Bootstrapping method of determining confidence intervals, and the 


differences were negligible. It is also worth noting that the input, Time Between 


Simulation Results Output, allows the user to record the results at any point during the 

















deployment. This gives information about the variability of the coverage (i.e. how 
coverage early in the deployment may compare with that later in the deployment, 


depending on initial conditions assumed). 
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IV. ANALYTIC MODEL 

An analytical model for a single platform has been developed, see Gaver, Jacobs, 
and Stoneman (1998) and Appendix A, that can be used to augment and assist in 
verifying the results of MASS. Such models can efficiently highlight sensitivities to 
basic model components (distributions of times to failure and times to repair, operational 
tactics) as a preliminary to more elaborate modeling and actual field testing. 

The first analytic model is a single-platform model. It assumes that mission- 
affecting failures occur according to a Poisson process with rate A, or, equivalently, that 
time to failure is exponentially distributed. The platform will return to base when a 
mission-affecting failure occurs. Additional failures that occur during the return add to 
the repair time. The repair times are independent and have a mean I/u. It is interesting 
and important that, for very long deployments, the only dependence on the repair time 
distribution is through its mean. This feature may change if one maintenance facility 
serves several platforms, and congestion and delays occur. 

The formula for long-run proportion of time on station from the analytical model 


1S: 


Es 
LU 
Where T = Ingress/Egress Time 
S = On Station Time 
D = Additional On-Deck Time (Tum, Logistics, etc.) 


It is notable that the expression for ETOS depends only on the expectations 


(means) of the repair times, and is insensitive to any other features of the repair time 
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distribution. Such insensitivity does not extend to times to failure (here exponential, but 


not necessarily so in practice); see sensitivity studies conducted in Chapter V, Section B. 


The inputs to MASS are, in this case, the following: 


Numerical [lustration 


// Number of Platforms 

// Number of Ground Maintenance Paths 

// Maximum Number of Platforms Flying 

// Ingress/Egress Time (in hours) 

// Scheduled On-Station Time (in hours) 

// Flight Time (in hours) Before Refuel 

// Transit Time (in hours) to Refueling Orbit 

// Time (in hours) Required to Refuel Platform 

// Probability That a Failure is In-Flight Repairable 

// Mean Time (in hours) Between Mission-Affecting Failures 

// Mean Time (in hours) Between Non-Mission-Affecting Failures 
// Mean Time (in hours) For Ground Repair; 3 examples 

// Mean Time (in hours) For In-Flight Repair 

// Flight Time (in hours) Between Scheduled Maintenance Actions 
// Deployment Duration (in hours) 

// Logistics Delay Time (in hours) 

// Scheduled Maintenance Service Time (in hours) 

// Platform Turn-Around Time (in hours) 

// Time Gin hours) Between Simulation Results Output 

// Number of Simulated Deployments (replications) 


The highlighted values are those that are used in the analytic model. Note that 


three different mean repair times are used. The results below in Table 1 indicate that 


MASS accurately estimates ETOS as computed analytically, and appears trustworthy for 


further studies with different parameters. 


Analytic Result MASS ETOS (95% CI) 
0.59 ~ 0.59 


0.49 — 0.50 





0.42 - 0.43 


Table 1. Comparison of Analytic Model with MASS 
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One of the interesting facts not inherently obvious from these results is the effect 
of how MASS begins the deployment. The analytic model results here do not take into 
account any start-up factors; it “starts in steady state”. MASS, on the other hand, starts 
off by having an ETOS of 0 for at least the first ingress time and possibly more if the first 
platform fails before reaching it’s assigned station. This difference is negated by using a 
deployment of sufficient length to make the start-up effects unimportant. 2,160 hours (90 
| days) proves sufficient for this. MASS and the analytical model will tend to disagree 
more if deployment time is short, with the (present) analytical model ETOS results being 
somewhat greater than those from MASS runs starting with commencement of ingress 


time. 
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V. RESULTS AND ANALYSIS. IMPLICATIONS FOR FIELD TESTING. 
There are currently more than 20 inputs for MASS. Instead of discussing the 

sensitivity analysis for each, the most important inputs have been selected for 
examination. Also some of the assumptions made are examined for their effect on the 
simulation. 
A. PRE-POSITIONING OF PLATFORM 

For this analysis the platform was assumed to have an endurance of 16 hours. The 
three cases chosen for comparison are shown in Table 2 
| 

Time Time 

Cael | 


Cae | 4 | 8 
CaseIN | 6 










Table 2. Description of Positioning Cases. 


The other inputs, which remained constant, were as follows: 
Numerical Illustration 


4 // Number of Platforms 

4 // Number of Ground Maintenance Paths 

4 // Maximum Number of Platforms Flying 
50,000 // Flight Time (Gn hours) Before Refuel 

0 // Transit Time (in hours) to Refueling Orbit 

0 // Time (in hours) Required to Refuel Platform 


0 // Probability That a Failure is In-Flight Repairable 

40 // Mean Time (in hours) Between Mission-Affecting Failures 
50,000 // Mean Time (in hours) Between Non-Mission-Affecting Failures 
2 // Mean Time (in hours) For Ground Repair 

0 // Mean Time (in hours) For In-Flight Repair 

50 // Flight Time (in hours) Between Scheduled Maintenance Actions 
2,160 // Deployment Duration (in hours) 

0.5 // Logistics Delay Time (in hours) 

7 // Scheduled Maintenance Service Time (in hours) 

] // Platform Turn-Around Time (in hours) | 

2,160 // Time (in hours) Between Simulation Results Output 

50 // Number of Simulated Deployments (replications) 
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Note that 50,000 hours is used as an input for flight time before refueling and time 
between non-mission affecting failures. As discussed earlier, this ensures that neither of 
these events will occur. Also the length of deployment is 2,160 hours to negate the effect 
of initial conditions. Most importantly note that by using 4 platforms and 4 maintenance 
paths with a mean of 40 hours between mission-affecting failures and only 2 hours for the 
mean time to repair, there is never a shortage of available platforms. This enables the 
effect of positioning on ETOS to be seen clearly. 

The a between mission-affecting failures are independent and have an 
paren distribution with appropriate mean time to failure. The times for repairs also 
are independent and have an exponential distribution with appropriate mean repair time. 


The results are in Table 3 and Figure 2. 


| | -ETOS (95% Confidence Interval) | 








Table 3. ETOS Results From Changes in Positioning 
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Figure 2. ETOS vs. Ingress Time 
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The significant change of the ETOS values for these different cases illustrates the 
advantage of basing the platform as close to the surveillance region as possible. There 
are two main reasons for this. The first is that any failures that occur during the ingress 
phase cause the platform to return to base. Such a platform contributes nothing to the 
time on station, and a replacement is launched, as soon as the failure occurs, if available; 
but the replacement is also subject to failure, and so on. The second reason positioning is 
so important is the time it takes for the replacement to reach the surveillance region. 
When a platform leaves the surveillance region at its scheduled time, a replacement is 
planned to be there to take its place. However, when there is a failure on station, the time 
it takes se the replacement to arrive is lost coverage, unless redundant presence is 
scheduled. These situations are demonstrated by Case I, where there is no ingress time, 
and the squadron is able to cover the region essentially 100% of the time, given the 
failure and repair parameters assumed. 

Again, a limitation of this analysis is the assumption that the squadron 
experiences no loss of platforms. This is not realistic since platforms can be lost due to 
mechanical failure or pilot error. However, this does not affect the user’s ability to detect 
sensitivities in a platform’s capabilities unless the platform has been shown to have an 
extremely high incidence of losses. Of course if that is the case, then the platform should 
probably be subject to more testing before fielding. It is also reasonable to assume that, 
the closer the platforms are based to a surveillance region, the greater the danger of 
attrition of platforms and maintenance facilities. Attrition could be added to the 
simulation as a decreasing function of distance, so as to effectively model a trade-off 


between the two variables. Of course for real world operations, the platform would most 


27 





likely be based at currently established sites, where attrition by the enemy is not a strong 
factor, and it is simply a matter of choosing the closest available location. 
B. TIMES BETWEEN MISSION-AFFECTING FAILURES 

One area of special interest that would seem to be a logical place to affect a 
platform’s suitability are the random times between failures. During the development 
phase of a platform, the company manufacturing the platform has the best chance to 
lengthen the time between failures through design and choice of parts to be used. It will 
then be a question of cost versus return. During the operational phase, a platform will be 
somewhat limited in its ability to increase the time between failures, but this may be 
possible by changing the way in which the platform is used. 

The question in both cases mentioned above leads to the comparison of ETOS 
with various mean times between mission-affecting failures. In this example the times 
between mission-affecting failures are independent and have an exponential distribution 
with appropriate mean time to failure. The times for repairs are also independent and 
have an exponential distribution with appropriate mean time to repair. The inputs to 
MASS for these runs were as follows with the inputs to be varied in boldface: 

Numerical Illustration 

// Number of Platforms 

// Number of Ground Maintenance Paths 

// Maximum Number of Platforms Flying 

// Ingress/Egress Time (in hours) 

// Scheduled On-Station Time (in hours) 
0,000 // Flight Time (in hours) Before Refuel 

// Transit Time (in hours) to Refueling Orbit 

// Time (in hours) Required to Refuel Platform 

// Probability That a Failure is In-Flight Repairable 
20,40,60 // Mean Time (in hours) Between Mission-Affecting Failures 


50,000 // Mean Time (in hours) Between Non-Mission-Affecting Failures 
2 // Mean Time (in hours) For Ground Repair 


Coo omwmoh fh hPL 
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0 // Mean Time (in hours) For In-Flight Repair 


50 // Flight Time (in hours) Between Scheduled Maintenance Actions 
2,160 // Deployment duration (in hours) 

0.5 // Logistics Delay Time (in hours) 

7 // Scheduled Maintenance Service Time (in hours) 

l // Platform Turn-Around Time (in hours) 

2,160 //Time (in hours) Between Simulation Results Output 

10 // Number of Simulated Deployments (replications) 


Note that when the mean time between mission affecting failures is equal to 40 
. the inputs are the same as Case IJ in Chapter V, Section A. The results can be seen in 


Table 4 and Figure 3. 


ETOS (95% CD) 


0.81 — 0.82 





0.89 —0.91 
0.93 — 0.94 


Table 4. ETOS Results From Change in Time Between Mission-Affecting Failures 
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Figure 3. ETOS vs. Mean Time Between Mission-Affecting Failures 


It is interesting to see how the simulation reacts to the changes. The increase in 


ETOS when the mean time was increased from 20 to 40 is a significant improvement. 


29 





However the increase of mean time from 40 to 60 has a diminished return rate. It would 
now be a matter of determining the cost associated with accomplishing the increases to 
determine if they would be worthwhile. This cost could be the result of improving 
platform reliability, shortening maintenance times, or both. 

C. TIME TO REPAIR 

Much attention has been paid to maintenance’s ability to affect the suitability of a 
platform. One simple way for the maintenance facility to increase a platform’s suitability 
should be to decrease the amount of time a platform is being served by decreasing the 
mean time to repair. 

The comparison of ETOS with various mean times to repair should illustrate the 
above point. In this example the times between mission-affecting failures are 
independent and have an exponential distribution with appropriate mean time to failure. 
The repair times are also independent and have an exponential distribution with 
appropriate mean time to repair. The scheduled maintenance service time is a constant. 
The inputs to MASS for these runs were as follows with the inputs to be varied in 
boldface: 

Numerical Illustration 
// Number of Platforms 
// Number of Ground Maintenance Paths 
// Maximum Number of Platforms Flying 
// Ingress/Egress Time (in hours) 
// Scheduled On-Station Time (in hours) 
0,000 // Flight Time (in hours) Before Refuel 
// Transit Time (in hours) to Refueling Orbit 
// Time (in hours) Required to Refuel Platform 
// Probability That a Failure is In-Flight Repairable 
// Mean Time (in hours) Between Mission-Affecting Failures 


50,000 // Mean Time (in hours) Between Non-Mission-Affecting Failures 
2,4,6 _ /{ Mean Time (in hours) For Ground Repair 


CoOoonws fp fH 
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0 // Mean Time (in hours) For In-Flight Repair 


50 // Flight Time (in hours) Between Scheduled Maintenance Actions 
2,160 // Deployment Duration (in hours) 

0.5 // Logistics Delay Time (in hours) 

7 // Scheduled Maintenance Service Time (in hours) 

l //{ Platform Turn-Around Time (in hours) 

2,160 // Time (in hours) Between Simulation Results Output 

10 // Number of Simulated Deployments (replications) 


Note that when the mean time to repair is equal to 2 the inputs are the same as 


Case IJ in Chapter V, Section A. The results can be seen below in Table 5 and Figure 4. 





ETOS 





2 4 6 


Mean Time To Repair 


Figure 4. ETOS vs. Mean Time to Repair 


The simulation does not react much to the changes. There are two likely reasons 
for this lack of effect. The squadron is equipped with 4 platforms and 4 maintenance 
paths, and each platform has a mean time between mission-affecting failures of 40 hours. 


In this case, the probability a platform will have a mission-affecting failure during its 


3] 





ingress or on-station time is 1 — e’”° 


= 0.26. So, with 4 platforms, the squadron will 
almost always have a platform available for launch, and the repair time, unless extreme, 
does not effect ETOS. 

It should be mentioned here that 4 maintenance paths for 4 platforms is an 
extreme number. It is unlikely that a maintenance facility would be able to handle the 
entire squadron being “down” at one time. This input for maintenance is used throughout 
this thesis for results that are not influenced by ground practices. In this case, it is 
worthwhile to see what happens with a smaller support structure. However, just one run 
confirms that there 1s not much effect in changing the maintenance structure. Using the 
inputs from above with a mean time to repair of 6 hours, and now only 1 maintenance 
path, the ETOS decreases only slightly to 0.87 (0.86 — 0.88 for the 95% CI). So, for these 
types of changes in a squadron’s support structure, it can be seen that there is not much 
“bang for the buck”. 

Now it should be confirmed that, with a smaller number of platforms, the ETOS 
will be affected by changing the mean time to repair. Once again the times between 
mission-affecting failures are independent and have an exponential distribution with 
appropriate mean time to failure. The times for repairs are also independent and have an 
exponential distribution. The inputs to MASS for these runs were as follows with the 
inputs to be varied in boldface. Note that the first three inputs have changed from the 
previous analysis. 

Numerical [lustration 
// Number of Platforms 
// Number of Ground Maintenance Paths 


// Maximum Number of Platforms Flying 
// Ingress/Egress Time (in hours) 
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// Scheduled On-Station Time (in hours) 

// Flight Time (in hours) Before Refuel 

// Transit Time (in hours) to Refueling Orbit 

// Time (in hours) Required to Refuel Platform 

// Probability That a Failure is In-Flight Repairable 

// Mean Time (in hours) Between Mission-Affecting Failures 

// Mean Time (in hours) Between Non-Mission-Affecting Failures 
// Mean Time (in hours) For Ground Repair 

// Mean Time (in hours) For In-Flight Repair 

// Flight Time (in hours) Between Scheduled Maintenance Actions 
// Deployment Duration (in hours) 

// Logistics Delay Time (in hours) 

// Scheduled Maintenance Service Time (in hours) 

// Platform Turn-Around Time (in hours) 

// Time (in hours) Between Simulation Results Output 

// Number of Simulated Deployments (replications) 


The results can be seen in Table 6 and Figure 5. 





2 4 6 


Mean Time To Repair 


Figure 5. ETOS vs. Mean Time To Repair 
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It can be seen that there is now a noticeable decrease in ETOS when the mean 
time to repair is increased. The smaller number of platforms increases the impact of the 
maintenance facility operations. Simply by increasing the number of platforms to 3, and 
using the worst case of 6 hours for the mean time to repair, the ETOS is raised back to 
0.85 (0.84 — 0.86 for the 95% CI). The interaction between the number of platforms and 
the maintenance support on ETOS is an interesting problem. The costs associated with 
increasing the number of platforms, or improving the maintenance support, will influence 
a decision concerning the most effective way to improve ETOS. 

D. SENSITIVITY TO VARIATION OF TIME-TO-FAILURE DISTRIBUTIONS 

Any aac of distributions may be suitable for modeling the times between 
failures on the platform. In the Predator model by Post and Warner, the times were 
assumed to have an exponential distribution. This is a conventional assumption for 
reliability models; it is a satisfactory and convenient first assumption but not a universal 
natural law. In this section, other distributions for the time between mission-affecting 
failures are examined for impact on ETOS. A mission-affecting failure is defined as a 
failure that requires an immediate return to base. The repair times are still independent 
and have an exponential distribution with appropriate mean time to repair. The 
simulation inputs were as follows: 

Numerical Illustration 

// Number of Platforms 

// Number of Ground Maintenance Paths 
// Maximum Number of Platforms Flying 
// Ingress/Egress Time (in hours) 

// Scheduled On-Station Time (in hours) 
0,000 // Flight Time (in hours) Before Refuel 


// Transit Time (in hours) to Refueling Orbit 
// Time (in hours) Required to Refuel Platform 


COWARD AHNA 
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0 // Probability That a Failure is In-Flight Repairable 

40 // Mean Time (in hours) Between Mission-Affecting Failures 
50,000 // Mean Time (in hours) Between Non-Mission-Affecting Failures 
2 // Mean Time (in hours) For Ground Repair 

0 // Mean Time (in hours) For In-Flight Repair 

50 // Flight Time (in hours) Between Scheduled Maintenance Actions 
2,160 // Deployment Duration (in hours) 

0.5 // Logistics Delay Time (in hours) 

7 // Scheduled Maintenance Service Time (in hours) 

] // Platform Turn-Around Time (in hours) 

2,160 // Time (in hours) Between Simulation Results Output 

50 // Number of Simulated Deployments (replications) 








These inputs remained constant for all model runs. Only the failure time distribution 
changed. See Appendix B for the derivation of the formulas used. The results are 


summarized in Table 7. | 


Time until next mission affecting failure | ETOS (95% 
| Exponential | CC—C—~dECSCC lambda) *(-inU) 0.67 - 0.68 


Weibull sqrt(2 * lambda) 0.5 ((-1/alpha)*InU)‘1/beta 0.51 —0.52 
Sculptured | .5 * (1Aambda - 1) 4 -InU *(1+(alpha*-InU)) 0.51 - 0.52 
(a) 


Sculptured | (1/24)*(1Aambda - 1) -InU * (1 + (alpha *-InU%3)) 0.28 - 0.30 
(b) 


e lambda = failure rate = 1/40 
e U=random uniform variable between (0,1) 










CT) 









Table 7. Time-To-Failure Distributions and the Resulting ETOS 


The two sculptured distributions were created using a technique described by Gaver 
(1983). They start with exponential distributions but rearrange the distributional form in 
a manner convenient for simulation. The shaping functions tend to leave the 
distribution’s relative shape for smaller values alone, but increase the probability of large 
values. This creates a distribution that is nearly exponential near the origin, but has an 
exaggerated long right tail (representing missing data, outliers, etc.). The net effect is to 


produce relatively more short/small times to failure than the exponential, and 
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correspondingly relatively fewer, but longer, times, so arranged that the mean is the same 
as in the exponential submodel (first line in Table 6). 

It is significant to note how the Effective Time on Station (ETOS) ts affected by the 
different distributions. 100,000 random uniform numbers in the range (0,1) were created 
in the statistical package S-Plus and put in the above formulas to confirm that all four 
distributions have the same mean value. Table 8 shows that they do have the same mean 
with any small differences due to sampling error. The confidence intervals were 


calculated using the normal assumption. 


Sculptured (b) | = 40.06 39.42 — 40.70 











Table 8. Mean Values of the Time-To-Failure Distributions 


It turns out that the ETOS may be generally dependent on, or sensitive to, the shape 
of the distribution of the times to failure. The ETOS produced by the Weibull 
distribution is quite comparable to the ETOS produced by the Sculptured (a) distribution. 
The Sculptured form (b) yields a significant drop in ETOS. 

Why did these different distributions produce the results that they did? In general, 
we make the reasonable assumption that the expected or mean time between mission 
affecting failures is greater than the flight time of our platform. Since the time until 
mission affecting failure is an independent draw each time the Platform launches, only 


failures that will occur during that flight can affect the ETOS. Therefore, only the part of 
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the distributions that produce values less than the flight time of 16 hours are a factor in 
the simulation. 
To show how the four distributions compare in producing failure times of less 
than the flight time of 16 hours, 10,000 additional random uniform numbers from (0,1) 
were created. They were then used in the formulas given in Table 7, and the number of 
times the resulting value was less than 16 (the sortie duration) was totaled. It can be seen 
in Table 9 that the distributions that are more likely to produce a failure in the first 16 
hours (during the flight) have a lower ETOS. 
Distribution ETOS (95% CI) Number of Failure Times < 
16 (out of 10,000) 
0.66 — 0.68 334] 
Weibull 0.51 — 0.52 5953 


Sculptured (a) 0.51 - 0.52 5893 
Sculptured (b) 0.28 - 0.30 8254 


















Table 9. Sample Failure Times 


These results are interesting because they point out how the overall model 


depends on basic assumptions: if there is a propensity for failure before the planned 





cycle time that is relatively high compared to an exponential with the same mean, 
adoption of the exponential model is overly optimistic. However, the alternative 
distributions explored in this thesis have not been chosen to actually model any real data. 
One other possibility that was not tested is a bimodal distribution. It has been ceed 
that the first hour is a likely time for failures because of a likely high number of take-off- 
related failures. After that, failures may occur less frequently until the platform returns 
when the landing process may again induce failures. Then it is likely that there are a high 


number of landing-related failures. This topic presents an area for further study. Actual 
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field test data, or the equivalent, on times to failure are essential in order to obtain valid 
results. 
E. LAUNCHING WITH NON-MISSION-AFFECTING FAILURES 

A non-mission-affecting failure has already been defined as a failure that does not 
affect the platform’s ability to complete its mission. For this suitability simulation, such 
failures only affect (increase) the time a platform spends in maintenance, thereby 
reducing ETOS. Originally, the simulation forced a platform to have all repairs 
completed before it was allowed to launch again. However, the assumption can be made 
that a platform will not enter ground maintenance unless it either has a mission-affecting 
failure, or it needs scheduled maintenance. So a platform with only non-mission- 
affecting failures will be sent directly to the queue to be Jaunched when needed; the turn- 
around time is assumed ‘to be 0. This assumption was made so that the simulation will 
defer repairing non-mission-affecting failures. The non-mission-affecting failures will 
not be repaired until the platform enters maintenance for one of the other two cases. 

The following inputs were used for model runs comparing the original MASS 
simulation to the variation that allows the platform to launch with non-mission affecting 
failures. In this example, the times between mission-affecting and non-mission-affecting 
failures are independent and have an exponential distribution with appropriate mean 
times to failure. The repair times are also independent and have an exponential 
distribution with appropriate mean time to repair. The inputs to be varied are in boldface. 


Numerical [lustration 


1,4 // Number of Platforms 

1,4 // Number of Ground Maintenance Paths 
1 // Maximum Number of Platforms Flying 
3 // Ingress/Egress Time (in hours) 
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14 // Scheduled On-Station Time (in hours) 

50000 // Flight Time (in hours) Before Refuel 

0 // Transit Time (in hours) to Refueling Orbit 

0 // Time (in hours) Required to Refuel Platform 

0 // Probability That a Failure is In-Flight Repairable 

25 // Mean Time (in hours) Between Mission-Affecting Failures 

5 // Mean Time (in hours) Between Non-Mission-Affecting Failures 
2 // Mean Time (in hours) For Ground Repair 

0 // Mean Time (in hours) For In-Flight Repair 


50 // Flight Time (in hours) Between Scheduled Maintenance Actions 
2160 // Deployment Duration (in hours) | 

0.5 // Logistics Delay Time (in hours) 

7 // Scheduled Maintenance Service Time (in hours) 

0 // Platform Turn-Around time (in hours) 

2160 // Time (in hours) Between Simulation Results Output 

10 // Number of Simulated Deployments (replications) 


The results are summarized in Table 10. 


oom: a Aime ee Time Ose CI) Oe CT) 
[8 | 053-055 | 054-056 


Son: Som ns Sa Sere a oe 
ETT 79 OT 0169 0.70— 
po a 8 0190.21 | 020-021 





Table 10. ETOS Comparison for Change in Maintenance Process 


The difference between the simulations is negligible when comparing the long- 
run effective time on station. This is interesting because at first glance it might be 
expected that allowing the platform to launch with non-mission-affecting failures 
unrepaired would increase the effective time on station. However this change in policy 
does not change the /Jong-run amount of time a platform spends in maintenance. The 
platform will still have the same number of failures and scheduled maintenance activities 
as it had before. The different policy merely causes the platform to switch from many, 


short trips to the maintenance facility to fewer, long trips to the maintenance facility that 


39 











occur later in the deployment. In the long run this results in approximately the same 
effective time on station. However, the different pattern of up and down times is 
potentially of interest. 

This can then lead to more areas to explore. For example, short deployments 
could use this policy to increase their ETOS. Then, of course, the question will arise as 
to what constitutes a “short” deployment. Since the object is to keep the platforms away 
from the maintenance facility, without any computations, a suggestion would be to limit 
the deployment to avoid scheduled maintenance. If all 4 platforms have just come out of 
scheduled maintenance and 50 hours is kept as the time until scheduled maintenance, 
then this would allow for 200 total hours of flight time, a significant deployment even 
when overlapping of the flights is taken into account. One could also rigidly adhere to 
the assumption that non-mission-affecting failures do not affect the platform, and never 
repair them. However this is not realistic unless the situation is extreme: a major effort 
must be conducted during a short deployment, with catch-up later. 

F. REPAIR TIME / DOWN TIME PROCESSES 

In section C, it was shown that allowing the platform to launch with the non-mission- 
affecting failures did not increase the long-run effective time on station. The reason for 
this was that the total required ground maintenance time remained the same. The 
maintenance time remained the same because of the assumption made in the simulation 
that the repairs are made serially. Therefore, when a platform lands with two failures and 
the mean time for repair of each is two hours, the platform spends four hours to repair the 
two failures on average. If the platform has ten failures, then it will take approximately 


twenty hours to repair. 
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The assumption of serial repair is not necessarily an accurate one. There should be 
more than one person or team assigned to do maintenance, and several maintainers could 
possibly work on different failures in parallel. For example, platforms are worked on by 
different types of technicians who repair failures in each of their specialties. This is 
usually split into the type of system on the platform such as airframe, engine, electronics, 
and mission equipment, such as sensor packages. Even if a platform had multiple 
electronic failures, it would still be reasonable to assume that more than one person iS 
available in that specialty. 

If the repairs are not completed serially, then another way to model them must be 
found. One possibility is that the repairs are completed in parallel. A repair time for 
each failure is computed, and then the maximum (rather than the sum) of those numbers 
is used as the total repair time. This change to MASS is accomplished by examining each 
platform’s failures when they land. The failures’ repair times are compared and the 
maximum is determined. The failure with the corresponding maximum repair time 
remains, and all other failures are given repair times of 0.001 hours (this merely a 
technical programming device). The platform then continues through the standard 
maintenance flow. 

The following inputs were used for model runs comparing the original MASS 

“simulation to the variation that performs repairs in parallel. The times between mission- 
affecting and non-mission-affecting failures are independent and have an exponential 
distribution with appropriate mean times to failure. The repair times are also independent 
and have an exponential distribution with appropriate mean time to repair. The inputs to 


be varied are in boldface. Note that these are the same inputs used in Section E. 
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Numerical Hlustration 


1,4 // Number of Platforms 

1,4 // Number of Ground Maintenance Paths 
1 // Maximum Number of Platforms Flying 
3 // Ingress/Egress Time (in hours) 

14 // Scheduled On-Station Time (in hours) 
50000 // Flight Time (in hours) Before Refuel 


0 // Transit Time (in hours) to Refueling Orbit 

0 . // Time (in hours) Required to Refuel Platform 

0 // Probability That a Failure is In-Flight Repairable 

20 // Mean Time (in hours) Between Mission-Affecting Failures 

5 // Mean Time (in hours) Between Non-Mission-Affecting Failures 
2 // Mean Time (in hours) For Ground Repair 

0 // Mean Time (in hours) For In-Flight Repair 


50 // Flight Time (in hours) Between Scheduled Maintenance Actions 
2160 // Deployment Duration (in hours) 

0.5 // Logistics Delay Time (in hours) | 

7 // Scheduled Maintenance Service Time (in hours) 

0 // Platform Turn-Around time (in hours) 

2160 // Time (in hours) Between Simulation Results Output 

10 // Number of Simulated Deployments (replications) 


The results are summarized in Table 11. _ 


Platforms | Paths | Airborne | Time Time (95% CI) (95% CD 
i 4 8 | 053-055 | 071-072 | 
pe A eee sO 0076 | 
pod a 8 019 -0.21 | 0.23-0.24 | 











Table 11. ETOS Comparison for Change in Repair Time Calculations 


As expected, the ETOS increased when the time the platforms spent in repair was 
decreased. These results may be realistic for a small number of failures, but may be very 
optimistic for a large number of failures. The “real world” is probably somewhere in 
between the pure serial and pure parallel repair. So another way to formulate the problem 


iS aS a combination of the two: series and parallel. For instance, assume that 
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maintenance can repair no more than three failures at once. Use the maximum repair 
time of each group of three as that group’s total time and then sum the values from each 
group of three to get a total repair time for the platform. This will penalize a platform for 
having a large number of failures but not as severely as the straight serial repair. 

Careful study of actual operational test data that include durations and patterns of 
repair of multiple failures is necessary to credibly quantify the random total down time 
likely to be incurred in operations. Such data can also be used to devise optimal repair 
schedules, and study logistics effects. 

G. OPERATING IN DEGRADED CONDITIONS: WHEN NOT TO REPAIR 

In previous simulation runs the platform has been treated as one system that can fail 
repeatedly during a mission because of its complexity. In this section the platform is 
separated into _ subsystems: the platform (P), and the sensor (S). Each subsystem 1S 
assumed to have three levels of performance: Good, Medium, and Bad. MASS can then 
be used with only a few adjustments to track the amount of time a platform spends in 
each of the nine possible states over the course of a deployment. The above is still a 
simplified version of an actual and specific system. The analysis points the way to study 
the levels of degraded operations of a specific new, or upgraded, system. Sensible 
options are possible. 

MASS inputs used were the same as before, but the time between mission-affecting 
failures 1s now the time between platform failures, and the time between non-mission- 
affecting failures 1s now the time between sensor failures to various performance levels. 
In other words, the first mission-affecting failure causes the platform to go to the Medium 


state (PM). If there is a second mission-affecting failure, during the same flight, the 
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platform goes to the Bad state (PB). The non-mission-affecting failures work in the same 
manner on the sensor. The first non-mission-affecting failure causes the sensor to go to 
the Medium state (SM), while the second causes the sensor to go rs the Bad state (SB). 

Suppose it is decided that the platform will return to base if the platform goes to the 
Medium state (PM). Also, it could possibly go to the Bad state during the return trip 
(PB). The state of the sensor does not affect the mission. The sensor 1s assumed to be 
| reliable enough that the probability of it entering the Bad (SB) state before returning to 
base is minimal. The system will continue to operate in the Medium (SM) or Bad (SB) 
state, but its effectiveness (e.g. sweep width) is less in the Medium than in the Good state. 

Note that many different system control options can be investigated by the model. 
For instance, if operation in the sensor Medium (SM) state is sufficiently degraded it may 
be desirable to dispatch the system for repair when it reaches that state, particularly if the 
maintenance facility is unloaded (no queue delay). 

Data collected from these runs is the same as the original MASS, but the flight hours 
that a platform spends in each possible state combination is recorded also. All failures, 
both mission-affecting and non-mission-affecting, are repaired before the platform is 
launched again. The times between failures are independent and have an exponential 
distribution with appropriate mean time to failure. Repair times are also independent and 
rave an exponential distribution with appropriate mean time to repair. Repair times do 
not vary according to the type of failure (platform or sensor). Time on the ground is not 
considered to be in any relevant state. The following were the inputs for the simulation: 
Numerical [lustration 


4 // Number of Platforms 
4 // Number of Ground Maintenance Paths 
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4 // Maximum Number of Platforms Flying 

4 // Ingress/Egress Time (in hours) 

8 // Scheduled On-Station Time (in hours) 

50,000 // Flight Time (in hours) Before Refuel 

0 // Transit Time (in hours) to Refueling Orbit 

0 // Time (in hours) Required to Refuel Platform 

0 // Probability That a Failure is In-Flight Repairable 

40 // Mean Time (in hours) Between Mission-Affecting Failures 

25 // Mean Time (in hours) Between Non-Mission-Affecting Failures 
1.9 // Mean Time (in hours) For Ground Repair 

0 // Mean Time (in hours) For In-Flight Repair 

50 // Flight Time (in hours) Between Scheduled Maintenance Actions 
2,160 // Deployment Duration (in hours) 

0.5 // Logistics Delay Time (in hours) 

a // Scheduled Maintenance Service Time (in hours) 

l // Platform Turn-Around Time (in hours) 

2,160 // Time (in hours) Between Simulation Results Output 

10 | // Number of Simulated Deployments (replications) 


The ETOS is 0.89 as it was in section A for case II since changing the meaning of 
the mean time between non-mission-affecting failures does not affect ETOS. Table 11 


shows the average percentage of flight time spent by a platform in each state. 


of flight time in state 
ee ee 
| Bad,Medium | 
| CiBad, Bad | CO 
















Table 12. Percentage of Time Platform Spend in Each State 


These results are qualitatively predictable, but they can lead to some further 
analysis. The platform spends 93% of its flight time with the platform in the Good state 


(PG). This occurs because the platform has a larger expected time between failures than 
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the sensor, and because the platform returns to base as soon as the platform goes to a 
Medium state. This latter reason is obviously why no significant time is seen with the 
platform in Bad state (PB); it is assumed that maintenance is always beneficial, but this 
may not be true. The quality-of-maintenance effect should be field tested. 

The sensor also spent most of its flight time in a good state although not to the 
Same extent as the platform. Since the platform does not return to base with sensor 
problems, it did operate with the sensor in a medium state for 19% of the flight time. The 
simulation also represented the system as having a small percentage of time with a bad 
sensor. This could mean that it is a bad policy not to return to base as soon as sensor 
problems arise. However, it should be remembered that even though the percentage was 
very small it may still have been an overestimation of any sensor problems that affected 
the mission. The platform could easily have been already returning to base when the 
sensor went bad which would have counted in these calculations. 

The evaluation of the sensor effectiveness is accomplished by breaking down the 
time when the platform is on-station, or when the platform is in a Good state (PG). The 


percentages of on-station time that the sensor spends in its’ three possible states are 


shown in Table 12. 
Sensor State Average Percentage of 
73.0 
23.1 
3.9 













Table 13. On-Station Percentage of Time Sensor Operates 
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These results show that the platform is on-station for a small percentage of time 
with the sensor in a Bad state. This would not be an ideal situation. Depending on what 
the actual real-world meaning of the sensor states is, it could reasonably be recommended 
that the platform be recalled before the sensor is allowed to degrade to the Bad state; this 
could shorten the total down time and improve ETOS and the quality/effectiveness of that 
time. However farther field testing could be done to determine if there appears to be an 
optimal balance between sensor effectiveness and time on-station. 

H. DEPLOYMENT LENGTH 

The effect of deployment length has not yet been studied extensively. A 90 day 
(2160 hr) deployment has been used for all previous analyses to allow the simulation to 
reach a steady state. This is a lengthy deployment from which to expect continuous 
coverage, but it was done to eliminate any effects from the startup period of the 
deployment and facilitate the comparison of the effects of varying the input parameters. 
However, it is useful and realistic to see how the platform performs for shorter 
deployments. 

The inputs for this analysis are the same as those used in section A, case II. The 
deployment length is then varied to observe the affect it has on the ETOS. The times 
between mission-affecting failures are independent and have an exponential distribution 
with appropriate mean time to failure. The repair times are also independent and have an 
exponential distribution with appropriate mean time to repair. The inputs are as follows: 
Numerical Illustration 
// Number of Platforms 
// Number of Ground Maintenance Paths 


// Maximum Number of Platforms Flying 
// Ingress/Egress Time (in hours) 


HHL 
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50 


// Scheduled On-Station Time (in hours) 

// Flight Time (in hours) Before Refuel 

// Transit Time (in hours) to Refueling Orbit 

// Time (in hours) Required to Refuel Platform 

// Probability That a Failure is In-Flight Repairable 

// Mean Time (in hours) Between Mission-Affecting Failures 

// Mean Time (in hours) Between Non-Mission-Affecting Failures 
// Mean Time (in hours) For Ground Repair 

// Mean Time (in hours) For In-Flight Repair 

// Flight Time (in hours) Between Scheduled Maintenance Actions 


24,72,168,336,672 // Deployment Duration (in hours) . 


0.5 


// Logistics Delay Time (in hours) 

// Scheduled Maintenance Service Time (in hours) 

// Platform Turn-Around Time (in hours) 

//Time (in hours) Between Simulation Results Output 
// Number of Simulated Deployments (replications) 


The deployment duration inputs are highlighted to indicate that they are varied in the 


simulation runs. The number of -hours correspond with 1, 3, 7, 14, and 28 day 


deployments. The simulation begins with the launch of the first platform. The resulting 


ETOS are shown in Table 13 along with the ETOS from the steady state run done earlier. 


ETOS (95% CD) 


0.70 — 0.77 
0.83 — 0.87 


0.87 - 0.90 
0.88 — 0.90 
0.89 - 0.90 


0.89 - 0.91 





Table 14. ETOS Results From Different Deployment Lengths 


The results are very similar for the longer deployments, while the shorter appear 


to have significantly lower ETOS. However, close inspection shows that the results are 


comparable for all the deployments. Each deployment must have the initial ingress time 


counted against it when calculating ETOS. For these runs, that is four hours that are lost. 


48 





Table 14 shows what happens when those four hours are taken out of the denominator of 


the ETOS calculations; this is approximately equivalent to “starting in steady state.” 


0.89 —- 0.91 
0.89 — 0.91 





0.90 — 0.91 


Table 15. ETOS Results With Start-Up Costs Removed 


The results are much closer together now. The ingress time for the first Platform 
had a more significant impact on the shorter deployments than the longer ones, and by 
taking that out it can be seen that the simulation is reaching the steady state in a much 
shorter deployment than 90 days. 

It should be pointed out that the results from the shorter deployments are probably 
more realistic than the 90 day results. A single squadron is not expected to provide 
continuous coverage of a region for 90 days without some relief. However, if a user is 
simply looking for trends in the data to pinpoint areas to improve performance, the 90- 
day deployment should be used to decrease variability (see Figures 5 and 6). If a user 
wishes to use the output to state that a platform can achieve a specific ETOS, then there 
can be some problems with the simulation of the long deployment. Attrition might be 
more of a factor for a long deployment, and its effect is not modeled. The platforms 
could begin to show signs of wear and increase their failure rate, and the people operating 
the systems might show signs of “wear” (fatigue) themselves, and this would affect 


operations. 
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The user should also be wary of the short deployment results: the ETOS is not as 


good a summary of Actual Time On Station (ATOS); there tends to be substantial relative 


variability for short deployments. Figure 6 is a histogram of the ETOS results from the 





168 hour deployment. 
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Figure 6. Histogram of 168 hour deployment results 


Figure 7 1s a histogram of the results from the 672 hour deployment. 
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Figure 7. Histogram of 672 hour deployment results 


The mean of the results from both runs were comparable, but there is still a 
difference. The histograms show that the shorter deployment exhibits greater variability 
of estimated ETOS. This makes sense, because a few quick failures would have a larger 
effect on the ETOS from the shorter deployment than they would on the longer 
deployment. 

I. CONCLUSIONS AND RECOMMENDATIONS 

MASS, especially when extended as has been done in this thesis, is an invaluable 
tool for test planning, and for the evaluation of a platform’s suitability for different 
missions. The simulation can assist during the early acquisition process, when the 
government must decide whether or not to buy a particular system; the simulation can 
also assist in determining the most effective way to deploy a platform and a particular 


payload once it is in the field. 
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While this thesis concentrates on UAV operations, MASS (extended) can be used 
for operational analysis of a variety of platform types. The inputs to the simulation can 
be varied to determine which facets of a platform provide the most opportunity for 
improvement of capabilities. Some of the present assumptions may not apply to certain 
platforms, but it is still possible to look for trends in the results to aid in decision-making. 
The simulation outcomes suggest operational sensitivities that may be studied further 
during field experiments. The simulation should help to focus the experiments. 

MASS has room for enhancements. Many of the assumptions can be changed (or 
removed) to adapt the simulation to answer other questions. In the case of UAVs, 
allowing for losses of platforms in the course of the deployment is one obvious possible 
modification. Adapting the desired area coverage to less than 24 hours a day would open 
the simulation to more platforms. Caution must be used to avoid making MASS less 
generic, thereby limiting the range of its usefulness. Also it could easily reach a point 
where the inputs to MASS would become numerous enough to discourage some users. 

Versions of MASS are currently being used by IDA to analyze a number of 
programs, including DARK STAR UAV, GLOBAL HAWK UAV, and JSTARS 
(manned reconnaissance platform). The versions of MASS in this thesis add to what is 


being learned there and should provide impetus for further applications and future work. 
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APPENDIX A: ANALYTIC MODEL CALCULATIONS 


B is the time between takeoff and return to base. Then 


2X if X < T (fails on flight out) 
B= X+T if T<X<T+S (fails during surveillance time) 
2T +S if X > T +S (fails on flight back or does not fail) 


Thus, 
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Let A be the time on station 


0 if X < T (fails on flight out) 
A= X-T if T<X<T +S (fails during surveillance time) 
S if X > T + S (fails on flight back or does not fail) 


t — o eg [ ste*ds + Pail 


ty ty 
 caeicaianess: | be-w. 
rs > 
ees, oe 
[I I! 
% 
l 
S 
te | 
~~ 
| 
_ 
eb 
| ree! | 


The expected length of a cycle is 


55 








prob of failure 
during cycle 


The long-run proportion of time the platform is performing surveillance: 


i= 
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APPENDIX B: DERIVATION OF TIME-TO-FAILURE DISTRIBUTIONS 


Weibull 
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Use B = % for long-tailed (hyper-exponential) data 
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Sculptured (b) 


Y = x(t + ax? | 
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